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IS  AN  INFORMATION  ANALYSIS  CENTER^  OPERATED  BY  I  IT  RESEARCH  INSTITUTE 
UNDER  CONTRACT  TO  THE  ROME  AIR  DEVELOPMENT  CENTER^  AFSC. 
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Center  (RADC),  and  operated  by  IIT  Research  Institute  (HTRI).  DACS  serves  as  a 
central  source  for  current)  readily  usuable  data  and  information  concerning 
software  technology. 
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database  of  empirical  data  collected  on  the  development  and  maintenance  of 
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PREFACE 


This  is  one  of  a  series  of  data  publications  dealing  with  software 
development  data.  This  publication  provides  a  summary  of  software 
development  experience  data  collected  by  the  NASA  Software  Engineering 
Laboratory  (NASA/SEL)  at  Goddard  Space  Flight  Center  during  the  1976-1980 
time  frame.  Other  publications  in  this  series  will  describe  other  data  sets 
of  the  OACS  database. 

Graphical  summaries  are  provided  only  for  those  projects  which  contain 
the  most  complete  data.  Tabular  summaries  are  provided,  however,  for  all 
projects.  The  original  NASA/SEL  database  refers  to  42  projects  but  not  all 
are  included  in  this  publication  due  to  the  incompleteness  of  the  data  for 
some  projects.  Some  projects  were  well  underway  before  data  collection  was 
initiated  and  others  are  still  undergoing  development. 

This  document  provides  a  summary  of  the  results  of  the  data  collection 
effort.  No  attempt  is  made  to  analyze  the  data  for  quality  or  completeness. 
This  compendium  does  not  include  exhaustive  listings  of  data  set  contents; 
these  kinds  of  listings  are  available  from  the  Data  &  Analysis  Center  for 
Software  (DACS)— in  hard  copy  or  machine  readable  form— upon  request. 

This  graphical  and  tabular  data  may  prove  to  be  useful  to  software 
engineers  and  researchers  in  a  number  of  ways.  It  provides  software  project 
development  historical  data  which  may  be  useful  for  studies  of  software  cost 
estimation,  project  monitoring  and  software  quality.  It  may  be  useful  in 
determining  the  relative  usefulness  of  Modern  Progranming  Practices  in  the 
software  development  and  maintenance  processes.  Finally,  it  provides  a 
source  of  data  which  can  be  used  to  develop  and  validate  cost  and 
reliability  prediction  models  across  a  variety  of  projects,  environments, 
applications,  etc. 
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The  tabular  data  from  which  this  compendium  is  derived  was  printed 
directly  from  the  OACS  computerized  database  utilizing  customized  retrieval 
and  report  generation  software  developed  by  the  DACS  programming  staff.  This 
system  allows  the  generation  of  special  reports  wherein  the  data  is 
categorized  to  match  the  needs  of  the  user. 

The  user  is  cautioned  that  the  data  produced  in  this  publication 
reflects  the  results  of  the  data  collection  process  only;  it  does  not 
necessarily  reflect  the  actual  project  development  environment. 
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1.  INTRODUCTION 


1.1  Purpose  of  the  Data  Compendium 

The  purpose  of  this  Data  Compendium  is  to  disseminate  information  on 
the  software  engineering  database  originally  developed  by  the  NASA/SEL  at 
Goddard  Space  Flight  Center.  It  has  been  produced  by  the  DACS  operated  by 
IIT  Research  Institute  (IITRI)  under  contract  to  RADC.  It  is  to  serve  as  a 
software  engineering  reference  on  historical  software  development  data.  The 
NASA/SEL  data,  in  addition  to  other  data,  is  maintained  in  the  DACS  software 
experience  database  for  analysis  and  model  validation  purposes.  Individuals 
who  are  conducting  software  engineering  research  may  obtain  subsets  of  the 
NASA/SEL  data  from  the  DACS  in  hardcopy  and/or  machine  readable  form. 

1 .2  Contents  of  the  Data  Compendium 

The  remainder  of  this  section  briefly  describes  NASA/SEL 's  approach  to 
data  collection  and  outlines  the  categories  of  data  summarized  in  this 
publ i cation. 

Section  2  consists  of  detailed  summaries  of  the  data,  in  graphical 
form,  for  the  projects  which  contained  the  most  complete  data.  An  overview 
of  the  development  environment,  however,  is  given  for  all  projects.  This 
overview  was  developed  by  the  DACS  through  personal  interviews  and 
examination  of  General  Project  Sunmary  Forms  provided  by  NASA/SEL. 

Section  3  consists  of  a  set  of  three  tables  of  summary  data  and 
associated  narrative  describing  the  three  main  categories  of  data  available 
across  all  projects.  These  data  categories  are: 

•  Project  Development  Data 

•  Change  Error  Data 

•  Development  Methodology  Data 

Each  of  these  three  categories  of  data  is  characterized  by  its 
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association  to  the  problem,  the  people,  the  process,  the  product,  the 
resources,  and  the  tools.  Some  factors  may  fit  more  than  one  category  but 
are  listed  only  once. 

Appendix  A  provides  more  in-depth  information  about  the  terminology 
used  in  this  Data  Compendium.  The  definitions  were  either  provided  by 
MASA/SEL  or  extracted  from  the  DACS  Glossary  (1). 


1 .3  Data  Collection  Approach 


The  NASA/SEL  was  founded  in  1976.  The  primary  purpose  of  the  NASA/SEL 
is  to  study  the  impact  of  various  management  and  programming  methods  on 
software  development.  The  approach  of  the  NASA/SEL  has  been  to  monitor  and" 
measure  the  software  life  cycle  during  actual  systems  development  and  then 
to  subject  these  measures  to  quantitative  analysis.  One  significant  result 
of  these,  efforts  is  the  software  engineering  database  suimarized  in  this 
report.  The  database  consists  of  software  experience  data  collected  on 
NASA/SEL  software  development  projects  through  the  use  of  seven  forms 
completed  periodically  by  project  personnel. 

(1)  The  General  Project  Sunmary  Form  was  used  for  project 
classification  and  progress  evaluation.  It  was  filled  out  by  the 
project  manager  at  the  beginning  of  the  project,  at  the  completion 
of  each  major  milestone  and  at  project  end. 

(2)  The  Programmer/Analyst  Survey  Form  was  used  to  track  the 
components  within  a  system.  A  component,  in  this  context,  is  a 
processing  module  identified  by  its  function  or  a  named  common 
block  of  shared  data.  The  form  was  completed  for  each  system 
component  wiien  it  was  defined,  when  it  was  implemented  and 
whenever  a  major  modification  was  made. 

(3)  The  Component  Summary  Form  was  used  to  keep  track  of  modules, 
subroutines,  block  cotimon,  etc.  of  the  system.  It  was  filled  out 
when  the  component  was  defined,  when  it  was  completed  and  when 
major  irwdifications  were  made. 

(4)  The  Component  Status  Report  was  used  to  track  resource 
expenditures  by  component.  The  form  was  completed  weekly  by  each 
team  member  working  on  the  project. 
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(5)  The  Resource  Sumnary  Form  was  used  to  track  project  costs.  It  was 
completed  weekly  by  the  project  manager. 

(6)  The  Change  Report  Form  was  used  to  evaluate  the  impact  of  system 
changes  on  the  development  cycle.  It  was  completed  every  time  the 
system  was  changed  due  to  modifications  or  discovered  errors  in 
specification,  design  or  code. 

(7)  The  Computer  Program  Run  Analysis  Form  was  used  to  monitor 
computing  activity  during  systems  dev  “''pment.  An  entry  on  the 
form  was  made  every  time  a  computer  run  was  initiated. 

Detailed  instructions  for  completing  these  forms  are  given  in  reference 

(2). 

This  data  has  been  made  available  on  tape  to  the  DACS  for  the  purpose 
of  comparative  analysis.  The  project  described  in  this  publication  are 
primarily  in  the  area  of  ground  support  software  for  satellite  attitude 
control  programs.  The  principle  programming  languages  used  in  these  projects 
were  FORTRAN  and  Assembly,  and  the  main  development  computers  were  an  IBM 
360  and  a  POP  11.  The  projects  range  in  size  from  12  to  600  modules  and  each 
consist  of  from  1200  to  110000  delivered  lines  of  source  code. 

The  data  is  not  complete  for  all  projects  because  some  were  well 
underway  before  the  data  collection  effort  was  initiated  and  some  projects 
are  still  undergoing  development. 

1 .4  Data  Sets 

The  following  subsets  of  the  NASA/SEL  data  have  been  extracted  from  the 
NASA/SEL  database  and  put  into  graphical  and  tabular  form: 

General  Project  Information 

Delivered  Source  Lines  of  Code 

New  Lines  of  Delivered  Code 

Number  of  Components 

Number  of  Modules 

Number  of  New  Modules 
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Number  of  Pages  of  Documentation 
Months  of  Development  Time 
Manmonths  of  Development  Effort 
Number  of  Computer  Runs 
Number  of  Computer  Hours 
Number  of  Program  Changes 
Project  Schedul ing  Information 
Design  Start  and  End  Dates 
Code  and  Test  Start  and  End  Dates 
System  Test  Start  and  End  Dates 
Acceptance  Test  Start  and  End  Dates 
Cleanup  Start  and  End  Dates 
Code  Production  Hi  story 
End  of  Week  Date 

Cumulative  Number  of  Source  Lines  Developed 
Cumulative  Number  of  Components  Developed 
Cumulative  Number  of  Code  Changes 
Devel opment  Effort  by  Acti vi ty ,  Manhours 
Design:  Create;  Read;  Review 
Development:  Code;  Read;  Review 
Testing:  Module;  Integration;  Review 
Profile  of  Run  Purposes  and  Resul ts 
Profile  of  Types  of  Changes  and  Errors 
Chronological  Failure  Data 
Failure  Number 
Date  Detected 
Date  Corrected 
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Type  of  Failure  or  Change 

Origin  of  Failure 

Number  of  Nodules  Affected 

Calender  Days  Since  Last  Failure/Change 


2.  PROJECT  DATA  SUMMARIES 

This  section  presents  detailed  project  summary  data.  The  data  is 
presented  as  a  set  of  graphs.  Each  subsection  is  organized  by  project  code 
into  a  set  of  narrative,  tables  and  graphs.  The  project  codes  are  derived 
from  the  NASA/SEL  database.  They  have  been  retained  to  maintain  consistency. 
For  example.  Project  K  is  described  in  subsection  2.K  in  the  following 


manner: 

Title  Figure  # 

General  Project  Summary  (Text) 

General  Project  Information  (Table)  2.K-1 

Project  Scheduling  Data  (Gantt)  2.K-2 

History  of  Documented  Source  Code  Development  2.K-3 

History  of  Module  Development  2.K-4 

History  of  Changes  2.K-5 

Distribution  of  Development  Effort  by  Task  2.K-6 

Distribution  of  Computer  Riins.  by  Purpose  2.K-7 

Distribution  of  Computer  Runs  by  Results  2.K-8 

Distribution  of  Changes  by  Type  2.K-9 

Distribution  of  Errors  by  Type  2.K-10 

Distribution  of  Effort  Required  to  Isolate  Errors  2.K-11 

Distribution  of  Effort  Required  to  Resolve  Changes  2.K-12 


Distribution  of  when  Errors  En'  red  the  System  by  Phase  2.K-13 
The  following  paragraphs  provide  an  overview  of  the  information 
represented  by  the  figures.  The  General  Project  Summary  is  provided  for  20 
projects.  Some  detailed  project  development  data  isn't  provided  in  this 
section  because  it  is  incomplete.  The  reader  will  note  that  some  subsections 
of  this  section  have  been  omitted  or  merely  include  text  describing  the 
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general  project  development  methodology.  These  subsections  will  be  completed 
as  additional  data  becomes  available  to  the  OACS. 

Subsections  which  contain  only  a  general  project  summary  are:  2.1,  2.3, 
2.11,  2.16,  2.17,  2.18,  2.32  and  2.33.  Subsections  which  have  been  oninitted 
because  there  was  little  or  no  data  in  the  database  for  the  corresponding 
project  code  are  2.4,  2.9,  2.12,  2.13,  2.14,  2.20,  2.22-2.25,  2.27-2.31, 
2.34,  2.36-2.38,  and  2.40-2.42.  Available  data  is,  however,  presented  in 
tabular  form  in  Section  3. 

General  Project  Information  (Figure  2^.]^-l.) 

The  General  Project  Information  Table  describes  project  size  in  terms 
of  lines  of  source  and  object  code,  number  of  components  and  pages  of 
documentation.  Project  development  time,  effort,  computer  resources  and  the 
number  of  engineering  change  reports  are  also  presented.  Several  items  are 
not  taken  directly  from  the  database,  but  rather  calculated  from  data  items 
stored  in  the  database. 

Months  of  development  time  is  computed  from  the  project  development 
schedule  and  manmonths  of  development  effort  is  computed  using  personnel 
hours  expended  during  development  and  is  based  on  172  hours  per  one 
manmonth.  The  component  sizing  data  is  based  on  data  recorded  in  the 
component  information  file.  NASA's  estimates  for  object  code  are  calculated 
using  the  equations: 

Object  Code  =  2.8  x  (Source  Code)  for  projects  of  less  than  or  equal  to 

15000  lines  of  source  code. 

Object  Code  =  2.1  x  (Source  Code)  for  of  greater  than 

15000  lines  of  source  code. 

Project  Schedul  ing  Information  (Figure  2.l<-2) 

The  second  figure  on  the  first  page  contains  project  scheduling 
information  in  the  form  of  a  Gantt  Chart.  The  graph  was  produced  directly 
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from  data  stored  in  the  database.  Actual  start  and  end  dates  for  each  of  the 
phases  of  project  development  are  given  in  Section  3. 

History  of  Documented  Source  Code  Production  (Figure 

The  next  graph  presents  a  weekly  history  of  code  production.  The  graph 
represents  the  cumulative  number  of  source  Tines,  including  comments 
produced  during  the  given  time  period,  and  is  taken  directly  from  the 
project  history  file  in  the  database. 

History  of  Module  Devel opment  (Figure 

This  graph  presents  a  weekly  history  of  the  cumulative  number  of 
modules  produced  during  the  given  time  period.  The  project  history  file  was 
also  used  to  produce  this  graph. 

History  of  Changes  (Figure  ^.1^-^) 

This  graph  presents  a  weekly  history  of  the  cumulative  number  of 
changes  made  to  the  project  during  the  given  time  period.  Note  that  these 
are  changes,  not  change  reports.  One  or  more  changes  are  usually  made  for 
each  change  report  which  hasn^t  been  rejected.  Again,  the  project  history 
file  provided  a  direct  source  for  the  information  contained  in  this  graph. 

The  project  history  file  contains  complete  data  for  eight  projects.  In 
some  cases,  the  data  on  project  code  development  and  module  development  is 
incomplete  because  the  number  of  lines  of  source  code  may  not  have  been 
recorded  or  had  comnents  added  to  it  until  a  large  portion  of  the  project 
had  been  completed.  This  may  account  for  some  missing  data. 

Pi stribution  of  Devel opment  Effort  by  Task  (Figure  ^.K-6) 

This  figure  represents  the  number  of  hours  spent  on  each  task  in  each 
of  the  first  three  phases  of  development:  Design;  Code  and  Unit  Test;  and 
Integration  Test.  These  hours  are  calculated  by  suiming  the  weekly  reported 
manhours  for  each  task  as  recorded  in  the  database  for  each  component.  A  pie 
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chart  is  used  to  represent  the  proportion  of  effort  spent  on  each  task 
during  development.  The  total  number  of  hours  reported  for  these  activities 
does  not  equal  the  total  effort  as  recorded  in  the  General  Project 
Information  Table  because  effort  expended  in  Acceptance  Testing  and  Cleanup 
was  not  included  in  this  distribution.  The  tasks  which  are  included  in  this 
distribution  of  effort  are: 

•  Design  Creation 

•  Design  Read 

•  Design  Review 

•  Code  Development 

•  Code  Reading 

•  Code  Reviewing 

•  Module  Testing 

•  Integration  Testing 

•  Testing  Review 

Distributions  of  Computer  Runs  by  Purposes  and  Results 
(Tigures  2.ki7~and  2.l<-8)  - - 

The  next  two  figures  represent  computer  run  data;  the  first  displaying 
the  purposes  of  runs  made  during  development,  the  other  displaying  results 
of  those  runs.  In  each  case,  the  proportional  number  of  runs  in  each 
category  is  represented  by  a  pie  chart.  The  data  for  these  two  figures  is 
derived  directly  from  the  database  which  records  the  purpose  for,  and 
results  of,  most  runs.  The  total  number  of  purposes  or  results  may  be 
greater  or  less  than  the  number  of  computer  runs  recorded  because  some  runs 
may  have  more  than  one  purpose  or  result,  or  the  purpose  or  result  of  a 
particular  run  may  not  have  been  recorded.  The  purpose  of  each  computer  run 
has  been  arranged  according  to  the  following  categories: 

•  Uni t  Test 
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•  System  Test 

t  Benchmarlc  Test 

•  Maintenance/Utility 

•  Compflatlon/Assembly/LInk 

•  Debug  Run 

•  Other 

The  number  of  run  purposes  for  some  projects  may  be  greater  than  the 
number  of  runs  reported  because  a  single  run  may  have  more  than  one  purpose; 
i.e.,  unit  test  and  debug. 

The  results  of  a  computer  run  have  been  arranged  according  to  the 
following  categories: 

•  Good  Run 

e  Submit  Error 

•  JCL  Error 

•  Other  Set-up  Error 

e  Hardware  Error 

•  Software  Error 

•  Compile  Error 

•  Link  Error 

•  Execute  Error 

•  User  Generated  Message 

•  Ran  to  Completion 

The  number  of  run  results  for  some  projects  may  be  greater  than  the 
number  of  runs  reported  because  a  single  run  may  have  more  than  one  result; 
i.e.,  a  run  ran  to  completion  but  included  a  user  generated  error  message. 
Distribution  of  Changes  and  Errors  ^  Type  (Figures  2. 1^-9  and  2.K-10) 

These  two  graphs  are  derived  from  data  recorded  on  Change  Report  Forms. 
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They  sumnarize  the  types  of  changes  and  errors  encountered  during 
development.  Data  is  recorded  when  an  event  occurs  which  triggers  a  change 
in  the  source  code  of  a  project.  Since  not  all  changes  are  a  result  of  error 
correction,  the  number  of  changes  may  not  necessarily  equal  the  number  of 
errors.  Also,  since  not  all  changes  are  made  to  code  (some  are  made  to 
requirements  and  specifications)  the  number  of  change  reports  may  not  equal 
the  number  of  changes  as  recorded  in  the  General  Project  Information  Table. 
This  phenomenon  may  also  be  due  to:  1)  one  change  in  source  code  resolving 
several  change  reports;  or  2)  changes  not  actually  being  made  for  every 
change  report  produced.  The  types  of  changes  categorized  in  the  database  and 
the  general  categories  under  which  each  fall  are: 

•  Corrective  Changes 

-  Error  Correction 

•  Perfective  Changes 

-  Planned  Enhancements 

-  Improvement  of  Clarity/Maintainability/Documentation 

-  Improvement  of  User  Service 

-  Optimization  of  Time/Space/Accuracy 

•  Adaptive  Changes 

-  Implementation  of  Requirements  Change 

-  Adaption  to  Environment  Change 

•  Other  Changes 

-  Utility  for  Development  Purposes  Only 

-  Other 

Not  all  change  reports  categorized  a  change  according  to  the  preceding 
categories.  No  category  of  change  was  reported  on  many  of  the  change 
reports. 


2-6 


When  a  change  report  does  categorize  the  reason  for  a  change  as  an 
error,  the  type  of  error  is  recorded  as  one  or  more  of  the  following 
categories: 

•  Incorrect  or  Misinterpreted  Requirements 

•  Incorrect  or  Misinterpreted  Functional  Specifications 

t  Error  in  the  Design  of  Several  Components 

•  Error  in  the  Design  of  One  Component 

•  Misunderstanding  of  the  External  Environment 

•  Error  in  Use  of  Programming  Language/Compiler 

•  Clerical  Error 

•  Other 

Not  all  changes  categorized  as  corrections  had  the  error  type(s) 
recorded.  Also,  some  changes  were  made  to  correct  more  than  one  type  of 
error. 

Distributions  of  Effort  Required  to  Isolate  Errors  and  Resolve  Changes 
IFTgures  2  iTanOT-ID - 

The  next  two  figures  categorize  each  error  by  the  effort  required  to 
isolate  it  and  categorize  each  change  by  the  effort  required  to  implement 
it.  Pie  charts  are  used  in  each  case  to  visually  represent  the  percentages 
within  each  effort  category.  Due  to  reasons  similar  to  those  mentioned 
above,  the  number  of  changes  may  not  necessarily  equal  the  number  of  change 
reports  as  recorded  in  the  General  Project  Sumnary.  Change  effort  was 
recorded  according  to  one  of  the  following  four  categories: 

e  Less  than  One  Hour 

•  One  Hour  to  One  Day 

•  One  Day  to  Three  Days 

•  More  than  Three  Days 

Effort  to  make  a  change  was  not  recorded  for  all  change  reports  nor  was 
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it  recorded  for  all  changes. 

Distribution  of  When  Errors  Entered  the  System  by  Phase  (Figure  2.K-13) 

The  final  chart  categorizes  the  errors  by  the  development  phase  in 
which  it  was  suspected  that  the  error  was  introduced  into  the  system.  The 
possible  development  phases  in  which  the  error  originated,  either  through 
omission  or  incorrect  implementation  are  categorized  as  follows: 

•  Requirements  Definition 

•  Functional  Specifications 

•  Design 

•  Coding  and  Testing 

•  Other 

•  Can't  Tell 

The  category  “Can't  Tell"  implies  that  the  phase  in  which  the  error  was 
introduced  could  not  be  determined.  It  bears  repeating  that  not  all  change 
reports  had  the  source  of  the  error  recorded.  Also,  some  may  have  had  more 
than  one  reason  recorded. 

In  suttmary,  where  data  was  available,  it  was  included  on  the 
appropriate  graph.  Whenever  a  specific  classification  of  a  run  or  a  change 
does  not  appear  on  a  pie  chart,  then  it  can  be  understood  that  no  runs  or 
changes  of  this  type  were  recorded.  In  some  instances  where  percents  are 
included  in  the  pie  charts,  the  total  percent  may  be  more  or  less  than  100. 
This  error  is  due  to  the  rounding  of  fractional  percentages  to  the  nearest 
percent.  In  cases  where  data  necessary  to  complete  a  given  chart  was  not 
recorded  at  all,  this  status  is  indicated. 
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2.1  Project  1 


Project  1  consists  primarily  of  Assembly  Level  Code  (ALC)  as  used  on 
POP  11/70  systems  and  was  developed  to  support  conversion  of  existing 
graphics  support  software  for  use  on  the  POP  11/70.  Specification  for  the 
project  was  functional  and  procedural,  using  flowcharts  and  baseline 
diagrams  (tree  charts).  Design  was  accomplished  through  iterative 
enhancement  and  the  development  was  top-down,  using  structured  code  with 
simulated  constructs  (program  stubs).  Validation  and  verification  testing 
was  both  top-down  using  stubs  and  bottom-up  using  drivers,  and  code-review 
was  performed  by  the  programner's  peers.  One  programmer  and  one  librarian 
were  employed  during  development  with  one  additional  person  being  used 
during  the  design  phase. 

The  data  on  Project  1  consists  primarily  of  change  error  data;  other 
data  for  the  most  part  was  not  recorded.  As  a  result,  no  pie  graphs  could 
be  developed  for  this  project.  Data  which  is  available  is  summarized  in 
the  tables  included  in  Section  3. 
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2.2  Project  2 

Project  2  consists  primarily  of  FORTRAN  source  code  and  was  developed 
to  support  spacecraft  orientation  computation.  The  target  computer  system 
for  this  project  was  an  IBM  360.  The  system  was  constructed  with  an  overlay 
structure  of  20  segments  divided  into  two  independent  programs.  The 
specifications  for  this  project  were  functional  at  the  subsystem  level  and 
design  was  accomplished  using  top-down  techniques.  Development  was  also 
top-down  at  the  highest  levels,  where  no  specific  coding  techniques  were 
used,  with  iterative  enhancement  being  used  to  develop  subroutines. 
Validation  was  accomplished  by  walk-throughs  and  formal  testing  procedures 
at  the. design  level.  The  project  personnel  were  organized  into  a  team  of  six 
persons,  with  one  chief  programmer  and  one  librarian. 

The  data  on  Project  2  is  relatively  complete,  error  types  by  category 
being  the  only  data  not  recorded. 
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2.3  Project  3 

Project  3  is  an  interactive  program  developed  to  assist  management  in 
resource  allocation.  The  project  was  designed  and  developed  to  operate  on  a 
POP  11/70.  Specifications  for  the  project  were  written  in  English  text 
(non-formal)  at  the  system  level.  Design  and  development  were  top-down  at 
the  project  level  with  unstructured  FORTRAN  being  used.  Baseline  diagrams 
(tree  charts)  and  detailed  system/module  specifications  were  used  during 
design  and  development,  respectively.  Validation  testing  was  accomplished  in 
a  top-down  manner  using  program  stubs,  and  the  quality  of  the  code  by  review  at 
the  module  level,  and  by  walk-throughs  at  the  system  level.  One  programmer 
and  keypuncher  were  involved  in  development. 

Data  on  Project  3  consists  of  some  component  information  computer  run 
data,  and  a  limited  number  of  change  reports. 


2.5  Project  5 

Project  5  consjsts  primarily  of  FORTRAN  source  code.  Its  purpose  was  to 
compute  spacecraft  attitude  based  on  telemetry  data.  The  system  consists  of 
an  overlay  structure  of  nine  segments.  Specifications  for  this  project  were 
functional  at  the  module  level.  The  project  was  designed  in  a  top-down 
fashion  using  program  stubs.  Baseline  diagrams  (tree  charts)  were  used  to 
specify  the  system  design.  The  project  was  developed  through  iterative 
enhancement  using  program  stubs  with  no  specific  coding  standard  required. 
Validation  testing  was  top-down  using  program  stubs,  and  inspection  was 
accomplished  by  code  reading  and  walk-throughs,  as  in  the  design  and 
development  of  the  system.  The  personnel  were  organized  into  a  team 
structure  consisting  of  a  chief  programmer,  librarian,  and  from  three  to 
five  assistant  programmers. 

No  data  was  recorded  in  the  Project  History  File  to  determine 
documented  source  code  or  module  development  history. 
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2.6  Project  6 

Project  6  consists  primarily  of  FORTRAN  source  code  and  was  developed 
to  compute  satellite  orientation  from  telemetry  data.  The  system  was 
developed  on  an  IBM  360  for  real-time  operation.  Specifications  for  the 
project  were  both  functional  and  in  formal  English  text  at  the  system  and 
subsystem  levels.  Design  was  top-down  at  the  system  level,  with  baseline 
diagram  (tree  charts)  being  the  formalism  used.  Development  was  by  iterative 
enhancement  with  no  specific  coding  technique  required.  Top-down 
verification  using  stubs,  code  reading  and  walk-throughs,  at  the  module 
level,  were  the  techniques  employed  to  validate  the  system.  The  personnel 
were  organized  into  a  team  consisting  of  one  chief  progranmer,  two 
librarians,  and  up  to  six  other  progranmers  at  any  given  time. 

Data  on  project  6  is  relatively  complete  for  each  category  of  data. 
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HOURS  'fflAr  THIS  aiSTRIBUTTON  IS  SASEO  ON:  $076 .2 


rTGURE  Z.5~a 


jrsTsrauTiOM.  OF  ’urfoses  ^r  rcmjTss  luns  oisncsunen  of  urours  of  :snwrts  ?ijhs 

■OTAL  'O.NS  ’EPOSTTO:  0377  'TTAl  ?UNS  ;E?ORTc3:  .377 

m  PURPOSES  ~»AT  *»is  oisrai3ur.Qii  :s  jaseo  :n:  ’54<  iln  jesults  -h*t  his  oisTRuuniM  :s  jaseo  om: 


TOTAL  NUMBER  OF  CHANGE  REPORTS  FOR  PROJECT  6:  491 


OISTKIBUnON  OF  CHANGES  BY  TYPE 
CHANGE  AEPORTS  HiAT  THIS  OISTBIBUnON  IS  BASED  ON:  24 


ADAPTIVE  CHANGE  4S, 


=iGURE  2.5-9 


OrSTHIBUTIQN  OF  irPORT  TO  tSOUTE  iSRORS 
:^ANG£  REPORTS  THAT  THIS  OISTRIBimON  :S  BAScD  ON:  :22 


REVES  rOUHO  51 


OlSntlBUTION  OF  ERRORS  BY  TTPE 
aUNGE  REPORTS  THAT  THIS  OISTRIBUnON  IS  BASED  ON:  02 


OISTRIBunCM  OF  EFFORT  RECJIRED  TO  TESOL/E  ^RANGES 
CHANGE  REPORTS  *HAr  TIIS  OISTRISUTTON  IS  3ASEO  CN:  ’60 


FIGURE' 2:  S‘-i2 


DISTRIBUTION  OF  WHEN  ERRORS  ENTERED  THE  SYSTEM  BY  PHASE 
CHANGE  REPORTS  THAT  THIS  DISTRIBUTION  IS  BASED  ON;  13 


FIGURE  2.6-13 


2.7  Project  7 

Project  7  Is  a  utility  program  designed  to  read,  display,  and  perform 
calculations  on  a  data  set  to  determine  satellite  attitude.  The  program 
consists  of  approximately  two  thirds  FORTRAN  source  code  and  one  third 
Assembly  Language  Code  (ALC).  The  software  was  developed  on  a  POP  U/70 
for  use  on.  an  IBM  360.  No  overlays  were  employed  in  this  program. 
Specifications  for  the  project  were  in  English  text  which  described  the 
top-level  of  the  program  logic.  Design  was  top-down  in  nature  using  baseline 
diagrams  at  the  module  level.  Development  was  accomplished  through  iterative 
enhancement  of  subroutines  composed  of  code  which  employed  simulated 
constructs.  Top-down  testing  of  modules  was  used  in  validation. 
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FIGURE  2.7-2 
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SYSTEM  “EST 
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"GURE  2.7- 


TOTAL  NUMBER  OF  CHANGE  REPORTS  FOR  PROJECT  7:  55 


No  data  recorded  for 
Type  of  Changes 


No  data  recorded  for 
Type  of  Errors 


2.8  Project  8 

Project  8  is  an  attitude  determination  program  designed  to  execute  in 
real-time  on  an  IBM  360,  and  is  composed  primarily  of  FORTRAN  source  code. 
Extensive  amounts  of  code  were  reused  from  Project  3  in  the  development  of 
this  project.  The  specifications  for  this  project  were  functional  at  the 
system  level  and  procedural  at  the  subroutine  level.  The  system  was  designed 
and  developed  in  a  top-down  fashion  using  baseline  dia.grams.  A  program 
design  language  was  used  to  specify  the  subroutine  level  functions.  Testing 
of  the  system  was  top-down  at  the  system  level  and  specification-driven  at 
lower  levels  during  validation.  The  personnel  were  organized  in  a  team 
structure  consisting  of  a  team  leader,  whose  responsibilities  paralleled 
that  of  a  chief  programmer,  a  librarian,  and  three  programmers  in  addition 
to  the  project  manager. 

The  data  on  Project  8  is  relatively  complete  for  each  category  of  data. 
Note  that  2792  changes  are  recorded  under  General  Project  Information. 

The  source  of  this  number  is  the  NASA  estimated  statistics  file.  However, 
the  project  history  file  shows  only  415  changes  for  Project  3. 
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TOTAL  NUMBER  OF  CHANGE  REPORTS  FOR  PROJECT  8:  239 
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Type  of  Changes 
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2.10  Project  10 

Project  10  consists  primarily  of  FORTRAN  source  code.  The  purpose  of 
the  program  was  to  determine  spacecraft  attitude.  The  specifications  for  the 
project  were  formal  at  a  subroutine  level.  Design  and  development  were 
top-down  at  the  subsystem  level  using  iterative  enhancement  at  the  subroutine 
level.  Baseline  diagrams  and  Program  Design  language  (POL)  were  the 
specification  techniques  used  to  design  the  program,  and  structured  code  was 
used  in  its  development.  Validation  was  specification-driven  at  the  system 
level.  The  project  personnel  were  organized  in  a  structure  similar  to  a 
chief  progratimer  team  with  one  chief  progranmer,  three  librarians,  and  five 


programmers  in  addition  to  the  project  manager. 

Run  analysis  and  resource  expenditure  data  is  not  recorded  for  Project 
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FIGURE  2.10-1 
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"GURE  2.10-12 


2.11  Project  11 

Very  little  information  is  available  concerning  the  development  of 
Project  11  other  than  that  it  is  for  a  scientific  application  and  composed 
primarily  of  FORTRAN  source  code.  Data  collected  on  Project  11  includes 
development  schedule  infomation  and  run  analysis  data. 
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2.15  Project  15 

Project  15  is  a  FORTRAN  program  which  analyzes  FORTRAN  source  code.  It 


was  developed  and  is  operating  on  a  POP  11/70.  The  system  is  in  an  overlay 
structure  of  three  segments.  The  specifications  for  the  project  were 
detailed  in  the  form  of  English  text  at  the  system  level  and  with  80%  of 
the  modules  having  procedure  oriented  specifications,  and  20%  having 
formal  specifications.  Baseline  diagrams  were  used  in  the  top-down  design 
and  development  of  the  system,  but  one  component  was  specifically  designed 
and  developed  in  a  "hardest  first"  fashion.  Structured  code  was  also  used  in 
development.  In  the  validation  of  the  system,  flow  of  control  was  tested  in 
a  top-down  fashion  using  stubs  while  modules  were  tested  in  a  bottom-up 
fashion  using  drivers.  Inspection  was  accomplished  by  code  reading  within 
modules  and  by  walk-throughs  at  the  subsystem  level.  Only  one  programmer  was 
used  in  the  development  of  this  project. 

Data  on  Project  15  is  relatively  incomplete.  The  number  of  changes 
recorded  on  the  estimated  statistics  file  (23)  is  inconsistent  with  the 
number  of  changes  recorded  on  the  project  history  file  (245). 
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2.16  Project  16 

Project  16  consists  of  a  FORTRAN  program  developed  to  generate  monthly 
reports  containing  financial  data.  The  program  is  a  single  overlay  structure 
developed  on  a  POP  11/70  but  able  to  execute  on  either  a  POP  11/70  or  an  IBM 
360.  Functional  specification  were  employed  at  the  module  level.  Program  and 
module  design  was  accomplished  using  top-down  techniques.  Bottom-up 
techniques  were  used  to  develop  each  module,  with  iterative  enhancement  of 
blocks  of  source  code  within  modules.  Flowcharts  were  used  to  specify  the 
design  of  each  module.  Code  reading  and  specification-driven  testing  were 
used  to  perform  verification  and  validation.  Only  one  programmer  was 
assigned  to  this  project. 

Data  on  Project  16  consists  of  component  and  resource  expenditure 
data  only.  As  a  result,  no  pie  charts  were  develooed  for  this  project. 

2.17  Project  17 

Project  17  is  a  FORTRAN  program  designed  to  perform  image  data 
retrieval  from  mass  storage  and  assist  in  calculation  of  attitude 
determination.  The  specifications  for  the  project  were  functional.  Design 
and  development  of  this  project  was  accomplished  using  an  iterative 
enhancement  approach  with  no  specific  coding  standards  being  recuired. 
Walk-throughs  were  used  in  the  verification  of  the  oroject.  Only  two 
programmers  were  assigned  to  this  project  with  one  suboroinate  to  the  other. 

Data  on  Project  17  consists  of  computer  run  analysis  data  only. 

As  a  result,  no  pie  charts  were  develooed  for  this  project. 
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2.18  Project  18 


Project  18  is  an  interactive  system  designed  as  a  generalized  graphics 
package  to  display  data  generated  by  other  systems.  The  project  consists  of 
three  independent  programs  written  in  FORTRAN.  It  was  developed  and  operates 
on  a  POP  11/70.  The  specifications  for  the  project  were  functional.  Design 
was  accomplished  using  a  data  flow  technique  with  iterative  enhancement  which 
utilized  baseline  diagrams.  Development  was  top-down  with  iterative  enhancement 
using  structured  code.  Top-down  testing  and  code  reading  were  employed 
during  verification  and  validation.  The  two  programmers  used  on  this  project 
were  not  organized  in  any  specific  manner. 

Data  on  Project  18  consists  of  partial  scheduling  information  and  a 
small  sample  of  change  reports.  As  a  result,  no  pie  charts  were  developed 

for  this  project. 
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2.19  Project  19 

Project  19  consists  of  a  main  attitude  determination  program  and  four 
utilities  subordinate  to  it.  Approximately  two  thirds  of  the  project  is 
written  in  FORTRAN,  the  remaining  third  being  written  in  ALC.  The  project 
Mis  designed  to  operate  in  near  real-time  on  an  IBM  360.  The  specifications 
for  the  project  were  both  functional  and  in  English  text  down  to  the 
subsystem  level.  Design  was  accomplished  by  iterative  enhancement  using 
baseline  diagrams  and  a  POL.  Development  of  the  project  was  also  by 
iterative  enhancement  of  simulated  constructs  of  blocks  of  code.  Validation 
and  verification  was  accomplished  through  top-down  testing  of  modules  and 
specification-driven  testing  of  functions.  The  personnel  assigned  to  the 
project  were  loosely  organized  into  a  prograirming  team  with  one  task  leader, 
one  librarian,  and  seven  programmers,  in  addition  to  the  project  manager. 

Data  on  Project  19  is  relatively  complete  for  each  category  of  data. 
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vGURE  2.13-11 


2 .21  Project  21 

Project  21  is  a  FORTRAN  program  developed  to  compute  orbital 
parameters.  The  program  was  developed  and  designed  to  operate  on  an  I3M  360. 
The  specifications  for  this  project  were  functional  to  the  module  level. 
Top-down  design  and  development  to  the  subroutine  level  and  iterative 
enhancement  of  the  subroutines  were  the  technidues  emoloyed  during  develop¬ 
ment.  During  design,  flowcharts  and  baseline  diagrams  were  used  at  the 
top-level  of  the  system  and  a  POL  was  used  in  the  design  and  development  of 
the  entire  system.  Top-down  testing  using  stubs,  coae  reading  and 
walk-throughs  were  used  in  validation  and  verification  of  the  program.  In 
addition  to  the  manager,  the  personnel  were  organized  into  a  team  consisting 
of  one  task  leader  and  one  programmer  analyst. 

In  addition  to  general  project  information,  data  on  Project  21  consists 
of  component  information  and  change  report  data.  It  is  important  to  note  tha 
delivered  lines  of  source  code,  manmonths  of  development  effort  and  number 
of  changes  were  not  recorded  in  the  NASA  database. 
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SEUERAL  PROJECT  INFORMATTOM 
PROJECT  ;  a 
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DELIYCRCO  LINES  OF  SOURCE  CODE 
NEU  UNES  OF  SOURCE  CODE 
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NASA  ESTIMATE  OF  NElt  OBJECT  OJOE  115968 

NUMBER  OF  COMPONENTS  MO 
NUMBER  OF  NOOULES  MO 
NUraER  OF  NE'i  NOOULES  MO 


AVERAGE  COMPONENT  SIZE  (C0MMENTE3  SOURCE) 
NINIMUM  COMPONENT  SICE  .'COlWVTEa  SOURCE; 
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NUMBER  3F  CHANGES 

NUMBER  OF  S:HANG£  REPORTS  -.50 


FIGURE  2.21-1 


PROJECT  31 

ACTOAL  Cr/EIOPMENT  SCHE'ULE  3Y  ■'^KZS 


::oE  =N0  “rr 


75 


HAR  SEP 

'.376 


IGURE  2.21-2 
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No  data  recorded  for 

History  of  documented  Source  Code  Production 
History  of  Module  Development 
History  of  Changes 
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TOTAL  NUMBER  OF  CHANGE  REPORTS  FOR  PROJECT  21:  150 


No  data  recorded  for 
Type  of  Changes 


::rn!!3UT:c!(  of  iSrors  or 
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ERRORS  ENTERED  T 


RTS  THAT  THIS  DISTRIBUTIO 


This  page  was  intentionally  left  blank 
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2.26  Project  26 

Project  26  is  a  FORTRAN  program  developed  to  support  attitude 
determination.  The  target  and  development  computer  for  this  project  was  an 
IBM  360.  The  system  consists  of  six  independent  programs.  Both  functional 
and  English  text  specifications  were  utilized  to  the  subroutine  level. 
Top-down  design  was  utilized  and  iterative  enhancement  of  subroutines  was 
employed  during  development,  using  structured  code,  baseline  diagrams,  and  a 
POL.  Top-down  testing  and  structure  and  specification-dri ven  testing  were 
used  in  validating  the  system,  and  code  reading  and  walk-throughs  were  used 
during  software  inspection.  The  personnel  were  organized  into  a  team 
consisting  of  one  chief  programmer,  one  librarian,  and  four  other  programers 
in  addition  to  four  managers. 

Data  on  Project  26  is  relatively  complete  for  each  category  of  data. 
Note  that  the  number  of  changes  recorded  under  General  Project  Information 
(191)  from  the  estimated  statistics  file  is  inconsistent  with  the  number 
of  changes  recorded  under  History  of  Changes  (1323)  from  the  project  history 
fi  le. 
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SENQUL  PROJECT  INFQRmnON 
PROJECT  :  2S 


SIS 


OEUVERED  LIMES  Of  SOURCE  CODE  39S1S 

NEK  LIMES  OF  SOURCE  CODE  S19SO 

MASA  ESnMATE  Of  UOROS  Of  OBJECT  CQOE  137977 

NASA  CmHATE  Of  NEU  OSJOT  CODE  U0099 

NUMER  Of  COMPONENTS  3S1 
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MINIMUM  COMPONENT  SIZE  (COPMENTED  SOURCE!  19 
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OEVELOPMENT 

MONTHS  OF  3EYEL0PMENT  TIME  '-A  S 

.•4AMM0NTHS  OF  JEVELOPFEUT  ifFORT  39 

NUMBER  Of  COMPUTER  TUNS  "379 

SYSTEM  360-35  HOURS  327.0 

SYSTEM  360-75  HOURS  333.0 

POP  31-70  HOURS  0.0 

NUMBER  Of  CHANGES  391 

NUMBER  OF  OIANGc  REPORTS  41 3 


FIGURE  2.26-1 


PROJECT  35 

ACTUAL  OEVELOPMENT  3C.HE0ULE  3Y  ’HAScS 


FIGURE  2.26-2 
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“HOJECT:  2« 
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FIGURE  2.26-5 
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jMir  "Esr  i; 


OISTRIBimON  OF  RESUtn  OF  OOMFUTTR  RUMS 
*OTAL  RUMS  REPORTED:  032’ 

RUM  RESULTS  that  ~h:S  OISTSiaUTtOM  IS  BASED  OM 


"GURE  2.25-' 


FIGURE  2.25-5 


TOTAL  NUMBER  OF  CHANGE  REPORTS  FOR  PROJECT  26:  413 


aiSTHisunoN  of  changes  ar  nps 

CHANGE  REPORTS  THAT  THIS  OISTHIBUTIOS  IS  3ASES  ON:  MS 


=IGURE  2,25-9 


OISTRIBUnOM  OF  ERRORS  BY  TTPE 
CHANGE  REPORTS  THAT  THIS  OISTRIBUTTOM  TS  BASED  ON:  2'. 


PlINCnONAL  SPECIFICATTCNS  INCORRECT 
OR  HISINTESPRETEir  ZS 


HtSUNOERSTANOING  OF 
EXTERNAL  ENVIRONMENT  11 


■rGaP£  2.25-10 


OISTRIBUTTON  OF  EFFORT  TO  ISOLATE  ERRORS 
CHANGE  REPORTS  THAT  THIS  OISTRIBUTTON  CS  BASED  ON;  CCS 


■RJRE  TrIAN  ONE  OAT  IZ 


"GURE  2. 25-11 


OISTRIBUTTON  OF  EFFORT  REO'JIRED  '3  RESOLVE  CANGES 
HAMGc  reports  Hat  his  OrSTRiaUTTCJt  IS  based  TN: 


2 .32  Project  32 


Project  32  1s  a  FORTRAN  program  developed  to  function  as  a  development 
and  maintenance  tool.  The  development  and  target  computer  system  for  this 
project  was  a  POP  11/70.  The  specifications  for  this  project  were  in  English 
text  at  all  levels  of  detail.  Top-down  design,  iterative  enhancement  and 
"hardest  first"  methodologies  were  used  in  the  design  of  this  project 
while  baseline  diagrams  were  used  to  specify  the  design.  Top-down  and 
specification- driven  testing  and  code  reading  and  walk-throughs  were  used  in 
inspection  and  validation  of  the  project-  In  addition  to  a  manager,  the 
personnel  included  one  task  leader  and  one  programmer  analyst.  Very  little 
data  has  been  recorded  on  Project  32. 

2.33  Project  33 

Project  33  is  a  FORTRAN  program  developed  to  collect  and  format 
information  for  radio  transmission.  The  project  was  developed  on  a  POP  11/70 
Put  designed  to  operate  on  an  ISM  360.  Both  functional  and  English 
specifications  were  used  at  the  system  level.  The  system  was 
designed  by  iterative  enhancement  of  modules  and  develooed  oy  the  same 
technique.  Baseline  diagrams  were  used  in  the  design  of  modules.  Structured 
coding  technioues  were  also  employed.  Modules  were  inspected  by  cede  reading 
and  specification-driven  testing  was  used  in  the  veri fi cati on/ va 1 i aati on  o* 
the  system.  The  personnel  were  organized  into  a  team  consisting  of  a  manager 
and  two  programmers.  Only  component  information  data  is  is  available  for 
this  project. 

2.34  Project  34 

No  data  is  recorded  for  Project  34. 


2-78 


2 .35  Project  35 

Project  35  is  a  FORTRAN  program  developed  to  extract  data  from  an  input 
file  and  write  it  to  an  output  file  for  processing.  The  development  computer 
systems  were  the  IBM  360  and  the  POP  11/70,  but  the  system  was  designed  to 
operate  on  the  IBM  360.  Procedural  specifications  were  utilized  at  the 
system  level  to  specify  the  design  of  the  software.  Top-down  design  in  the 
form  of  baseline  diagrams  and  top-down  development  using  a  POL,  were  other 
technioues  employed.  Speci fi cation -driven  testing  was  used  to  test  the 
program,  and  coae  was  inspected  by  walk-throughs  and  code  reading.  The  two 
programmers  assigned  to  the  project  were  subordinate  to  one  supervisor. 

Data  on  Project  35  is  relatively  complete  in  each  category  of 
information.  However,  no  history  data  was  recorded  for  this  project. 
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No  data  recorded  for 

History  of  Documented  Source  Code  Production 
History  of  Module  Development 
History  of  Changes 
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PMUeCT:  IS 
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FIGURE  2.35-7 


=IGURE  2.55-i 


TOTAL  NUMBER  OF  CHANGE  REPORTS  FOR  PROJECT  35:  103 
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FIGURE  2.35-9 
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FIGURE  2.35-11 
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This  page  was  intentionally  left  blank 
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2.39  Project  39 

There  is  little  information  available  concerning  the  develooment  of 
Project  39  other  than  the  computer  used  was  an  IBM  360. 

Data  has  been  recorded  in  all  categorie'’  except  the  project's  source 
code  and  module  development  and  change  histories. 
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NIMBI  OF  COIVONENTS  7A 
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NIMBI  OF  NEW  MODULES  IS 


AVERA6E  COMNNENr  SIS  (COWENTtO  SOURCE) 
MINIMUM  COMFONENT  SIS  (COMIEHTIQ  SOURCE] 
MAXIMUM  COMPONENT  SIS  (COWCNTEO  SOURCE) 


scvaoPMENr 

MONTHS  OF  OEYELOPMENT  HME  IS. 5 

MANHONINS  OF  OEYELOPMENT  EFFORT  16 

NUMBER  OF  COMPUTER  RUNS 
SYSTEM  160-95  HOURS 
SYSTEM  160-75  HOURS 
POP  11-70  HOURS 

NUMBER  OF  CHANGES 

NUMBER  OF  CHANGE  REPORTS  17 

FIGURE  2.39-1 
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Fir^URE  2.39-2 
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No  Data  recorded  for 
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FIGURE  2.39-r 
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*OTAL  5UHS  REPORTED: 
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=IGURE  2.39-6 


TOTAL  NUMBER  OF  CHANGE  REPORTS  FOR  PROJECT  39: 


15 


aisTaiBunod  of  changes  31  typ? 

:hang£  reports  run  ™is  3ISTRIBUTT0N  is  SASED  CN:  is 
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3.  CROSS  PROJECT  SUM^RIES 


This  section  consists  of  a  set  of  three  tables  (3.1-1,  3.2-1,  and 
3.3-1)  of  summary  data  which  summarize  the  three  major  categories  of  data 
available  across  all  projects.  These  data  categories  are: 

•  Project  Development  Data 

•  Change  Error  Data 

a  Development  Methodology  Data 

Each  of  these  major  data  categories  has  been  further  subdivided  into 
subcategories  representing  phasing  and  scheduling  data;  human  and  machine 
resources;  project  size,  composition  and  development  history;  run  purposes 
and  outcomes;  and  finally,  the  distribution  of  when  errors  were  introduced 
into  the  software,  as  well  as  the  effort  reouired  for  correction. 

Unfortunately,  data  results  are  incomplete  for  a  number  of  pr<.,ects. 
This  is  because  some  projects  were  already  well  under  development  when  the 
data  collection  process  was  initiated.  Data  is  missing,  most  notably,  in  the 
computer  run  analysis  area  and  the  error  analysis  area. 

3 .1  Project  History  and  Development  Data 


The 

number  of  projects  for  which  data 

is  available  for  each  subcategory 

is  as  follows: 

I 

.  Project  Size 

19 

i  i 

.  Development  Time 

13 

Development  Effort 

IS 

III 

.  Development  Time  by  Phase/Development  Effort  by  Phase 

Design 

24/22 

Code  and  Test 

23/21 

System  Test 

22/20 

Acceptance  Test 

21/0 

3-1 


Cleanup 


18/0 

14 


IV.  Computer  Resource  Expenditures 
This  data  Is  sunmiarized  In  Table  3.1-1. 

3.2  Change  Report  Data 
Data  Is  relatively  complete  for  10  of  the  projects. 

The  number  of  projects  for  which  data  is  available  for  each  subcategory 


is  as  follows: 

I.  Mumber  of  Change  Reports  21 

II.  Distribution  of  Changes  by  Phase  10 

III.  Distribution  of  Why  Changes  Made  13 

IV.  Distribution  of  When  Error  Entered  18 

V.  Distribution  of  Effort  to  Resolve  Change  20 

This  data  is  suirmarized  in  Table  3.2-1. 


3.3  Development  Methodology  Data 

Data  is  relatively  complete  for  21  projects.  Software  development 
constraints,  as  well  as  the  Modern  Programming  Practices,  techniques  and 
tools  utilized  during  the  development  of  the  VASA  software,  are  listed  in 
the  following  key  to  Table  3.3-1. 

Table  3.3-1  contains  a  summary  of  these  special  environmental  factors 
by  project. 
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aiNtHAl  imojltl  UlVllOI’HtHI  OAIA  (I’AHI  1) 


lAuu  i  l  l  attiKAL  imiai  uiviioi'Hini  uaia  (hak 


lABlE  3.2  I  CHANGE  IN  EHHOK  IIAIA 


KEY  TO  SPECIAL  ENVIRONMENTAL  FACTORS 
WHICH  INaUENCED  PROJECT  DEVELOPMENT 


1.  A  special  display  requiring  new  or  complex  support  software,  acted  as  a 
constraint  on  development.  Y  =  Yes,  N  *  No 

2.  A  detailed  definition  of  operational  requirements  aided  project  development. 

Y  =  Yes,  N  =  No 

3.  The  existence  of  changes  made  to  the  operational  requirements  during  develop¬ 
ment  constrained  to  some  degree  the  development  of  the  project. 

N  *  Mo  constraint,  1  =  Very  little  constraint, 

5  represents  a  large  constraint 

The  project  was  designed  for  real-time  operation.  Y  =  Yes,  M  =  No 

5.  The  project  was  developed  with  a  constraint  on  the  program  processor 
memory  size.  N  =  No  constraint,  i  =  Very  little  constraint, 

5  represents  a  large  constraint 

5.  The  project  was  developed  with  a  constraint  on  the  ooeration  time  of  the 
project.  N  =  No  constraint,  1  =  Very  little  constraint, 

5  represents  a  large  constraint 

7.  The  project  was  the  first  software  developed  for  a  particular  computer 
or  operating  system.  Y  =  Yes,  N  *  No 

3.  The  project  was  developed  concurrently  with  ADP  hardware  necessary  for  the 
operation  of  the  software.  Y  =  Yes,  N  =»  No. 

9.  The  system  used  in  development  was:  Time-Sharing  =  7  or  Batch  =  3. 

10.  The  development  of  the  oroject  was  constrained  by  situation  of  develooers 

having  to  use  a  system  other  than  their  own.  Y  =  Yes,  N  =  No 

11.  The  project  development  took  place  at  the  operational  site.  Y  =  Yes,  N  =  Mo 

12.  The  develooment  and  target  computers  were  different.  Y  =  Yes,  M  =  No 

13.  The  develooment  of  the  project  took  Place  at  multiple  sites. 

"  =  Yes,  N  =  No 

U.  The  programmer's  level  of  access  to  the  computer  dialog  development. 

1  *  Very  limited  access,  5  =  Unlimited  access 

15.  Form  of  specifications. 

A.  Functional 
8.  Procedural 
C.  English 
0.  Formal 
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16.  OesS qn  techniques. 


A. 

Top-down 

D. 

Hardest  first 

B. 

Bottom-up 

E. 

Other  used 

C. 

Iterative  enhancement 

F. 

None  used 

17. 

Development  techniques. 

A. 

Top-down 

D. 

Hardest  first 

B. 

Bottom-up 

E. 

Other  used 

C. 

Iterative  enhancement 

F. 

None  used 

18. 

Coding  techniques. 

A. 

Simulated  construct 

B. 

Structured  code 

C. 

Other  construct 

0. 

None  Used 

19. 

Testing  techniques. 

A. 

Top-down  (stubs) 

0. 

Structure  driven 

B. 

Bottom-up  (drivers) 

E. 

Other  used 

C. 

Specification  driven 

F. 

None  used 

20. 

Inspection  techniques. 

A. 

Code  reading 

B. 

Walk-through 

C. 

Other  used 

0. 

None  used 

21. 

and 

22.  Design  and  development  formali 

sms. 

A. 

POL 

D. 

Baseline  Diagrams 

B. 

HIPO 

E. 

Other 

C. 

FI  owcharts 

F. 

None 
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APPENDIX  A 


Glossary  of  Terms 

This  appendix  contains  definitions  of  the  terms  used  in  describino 
the  NASA/SEL  software  engineerino  database.  It  has  been  comoiled  from 
references  (1)  and  (2).  The  major  objective  in  providing  this  glossary  is  to 
promote  consistency  in  terminology  usage  among  researchers  in  software 
engineering. 


Phasing  and  Scheduling:  All  of  the  activities  that  include  an  evaluation 

of  a  oroject's  reouirements ,  dividing  those 
requirements  into  specified  sets  of  goals,  and 
making  assignments  to  complete  each  set  of  ooals. 

Design:  A  description  of  how  software  will  be  oroducec 

to  satisfy  the  project’s  specifications. 


System  Test: 


Acceptance  Test: 


Cleanup 


Resource  Expenditures: 


The  process  of  trying  to  find  discrepancies 
between  the  system  and  the  original  objectives. 

The  testing  of  the  software  in  the  presence  of  the 
user  to  determine  if  it  meets  predetermined  user 
requi  rements. 

The  preparation  of  system  tapes,  formatting  of 
test  results,  completing  documentation,  etc. 
that  occurs  after  acceptance  tasting.  Mo 
testing  normally  occurs  durino  cleanuo. 

The  value  of  the  resources  consumed  in  the  com¬ 
pletion  of  a  project.  Those  resources  include 
human  resources  and  machine  resources.  Human 
resources  may  be  divided  into  three  cateoories: 
manaoement,  programmers,  and  clerical,  '^acnine 
resources  include  the  ccmouter  time  used. 


Design  Phase: 


Development  Phase: 


The  creation  and  recordino  of  the  desian,  including 
discussion  about  strateoy  with  peers  and  the 
creation  of  specifications  for  subcomponents  of 
the  current  component.  This  phase  also  includes 
a  review  of  decisions  made  durino  creation  and 
recording  of  the  design. 

The  development  and  recording  of  code  and  in-line 
comnent  based  on  the  design.  This  phase  includes 
the  modification  of  code  caused  by  design  changes, 
errors  found  in  testing,  and  a  review  of  the  work 
done  in  this  phase. 
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Testing  Phase: 


The  design  of  tests,  testing  strategies,  and 
the  running  of  such  tests,  for  each  module  and  the 
integration  of  the  modules  into  the  project. 


Module: 

Line  of  Code: 

Unit  Test: 

Maintenance: 

Utilities: 

Compile: 

Assemble: 

Link: 

Cebugging : 

3en  chmark : 

Successful  Run: 


A  program  unit  that  is  discrete  with  respect  to 
compiling,  characterized  by  lexical  binding, 
identifiable  proper  boundaries,  named  access  and 
named  reference.  The  word  module  may  apply  to  a 
subprogram,  program,  subroutine,  routine, 
function  or  macro. 

Seventy-two  character  card  image  of  source  code 
including  comments. 

Testing  of  a  program  segment  or  set  of  instructions 
treated  logically  as  a  whole. 

The  process  of  modifying  existing  operational 
software  while  leaving  it's  primary  function  intact, 
including  detection  and  correction  of  errors 
and  the  incorporation  of  modifications  to  add 
capabilities  and/or  improve  performance. 

Computer  programs  which  provide  special  services, 
such  as  preparing  program  deck  listings,  moving 
files,  creating  load  tapes  and  plotting  output 
results. 

To  translate  a  computer  program  expressed  in  a 
problem-oriented  language  into  a  computer- 
oriented  language. 

To  translate  a  set  of  some  language  statements, 
usually  the  computers  machine  language,  into  the 
computer’s  machine  code. 

To  establish  correspondences  within  a  set  of 
code  segments  which  satisfy  references  between 
segments. 

The  process  of  determining  wnether  or  not  errors 
exist,  attempting  to  isolate  the  source  of  a 
problem  and  finding  a  solution. 

A  standardized  computer  program  used  to  test  the 
processing  power  of  different  computers.  Input 
data,  computations  to  be  performed,  and  the  out¬ 
put  formats  are  specified  very  rigidly. 

A  program  execution  which  runs  to  completion  and 
produces  the  output  expected. 


Submit  Error; 


JCL  Error; 

Compile  Error; 

Setup  Error; 

Hardware  Error; 

Software  Error: 

Link  Error; 

Execution  Error; 

■Jser  Message  Error; 

Requirement  Definition : 
Functional  Specifications: 


Occurs  when  program  block,  or  complex  set  of 
code  is  improperly  executed  because  of  misunder¬ 
standing  involved  in  the  directions  for 
execution. 

An  error  occuring  during  the  use  of  the  Job 
Control  language  or  misuse  of  a  procedural 
operator. 

An  error  resulting  from  a  misunderstanding  of  how 
the  compiler  operates  or  an  error  resulting  from 
the  translation  of  a  high  order  language  to  a 
machine  based  language. 

Error  resulting  from  improper  ordering  of  cards, 
modules  or  program  blocks  in  a  job  deck,  or  in  use 
of  an  editor. 

Error  resulting  from  the  breakdown  or  mal •function¬ 
ing  of  the  physical  component  or  circuitry  in  the 
computer. 

A  discrepancy  between  a  computed,  observed  or 
measured  quantity  and  its  true  specified,  or 
theoretically  correct  value,  caused  by  deficiencies 
or  misinterpretations  of  design  criteria,  logical 
mistakes,  or  syntatical  mistakes. 

Error  resulting  from  the  linking  of  code  segments 
usually  involving  transfer  of  control,  label 
definition  and  location,  or  absence  of  a  referenced 
code  segment. 

Error  caused  by  improper  use  of  an  algorithm,  or 
imoroper  algorithm  for  data  supplied.  Program 
usually  terminates  tut  output  is  inaccurate. 

Occurs  when  a  run  is  terminated  by  the  user,  or 
programmer  when  an  error  is  discovered  in 
execution. 

A  statement  of  what  the  user  expects  the  system 
to  include  among  its  capabilities. 

A  set  of  functions  defining  the  output  for  any 
input,  emphasizing  what  the  program  is  to  do, 
rather  than  how  to  do  it. 

Time  involved  in  a  modification  to  design,  code  or 
documentation,  to  correct  an  error,  imorove  system 
performance,  add  a  capability,  or  implement  a 
requirements  change. 


Change  Effort: 


Design  Create  Phase: 

Design  Read  Phase: 

Design  Review  Phase: 

Development  Code  Phase: 

Development  Read  Phase: 
Development  Review  Phase: 

Test  j"^odule  Phase: 

Test  Integration  Phase: 
Test  Review  Phase; 
Dataset: 

Database: 


Writing  of  component  design. 

Reading  of  design  by  peer  to  look  for  errors. 

Formal  meeting  of  several  individuals  for  purpose 
of  explaining  design,  (management  review) 

Writing  executable  instructions  and  desk 
checking  program. 

Code  reading  by  peer,  similar  to  Design  Read. 

Management  review  of  coded  components,  similar 
to  Design  Review. 

Module  testing  -  test  run  with  test  data  on 
single  module. 

Integration  testing  of  several  components. 

Management  review  of  testing  status. 

Denotes  a  collection  of  data  from  a  source  (e.g., 
a  software  development  project). 

Denotes  a  collection  of  datasets  compiled  for 
analysis  purposes  (e.g.,  software  reliability 
analysis). 


